27 results
2 - Linear tools and general considerations
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 13-29
-
- Chapter
- Export citation
-
Summary
Stationarity and sampling
Quite generally, a scientific measurement of any kind is in principle more useful the more it is reproducible. We need to know that the numbers we measure correspond to properties of the studied object, up to some measurement error. In the case of time series measurements, reproducibility is closely connected to two different notions of stationarity.
The weakest but most evident form of stationarity requires that all parameters that are relevant for a system's dynamics have to be fixed and constant during the measurement period (and these parameters should be the same when the experiment is reproduced). This is a requirement to be fulfilled not only by the experimental set-up but also by the process taking place in this fixed environment. For the moment this might be puzzling since one usually expects that constant external parameters induce a stationary process, but in fact we will confront you in several places in this book with situations where this is not true. If the process under observation is a probabilistic one, it will be characterised by probability distributions for the variables involved. For a stationary process, these probabilities may not depend on time. The same holds if the process is specified by a set of transition probabilities between different states. If there are deterministic rules governing the dynamics, these rules must not change during the time covered by a time series.
In some cases, we can handle a simple change of a parameter once this change is noticed. If the calibration of the measurement apparatus drifts, for example, we can try to rescale the data continuously in order to keep the mean and variance constant.
1 - Introduction: why nonlinear methods?
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 3-12
-
- Chapter
- Export citation
-
Summary
You are probably reading this book because you have an interesting source of data and you suspect it is not a linear one. Either you positively know it is nonlinear because you have some idea of what is going on in the piece of world that you are observing or you are led to suspect that it is because you have tried linear data analysis and you are unsatisfied with its results.
Linear methods interpret all regular structure in a data set, such as a dominant frequency, through linear correlations (to be defined in Chapter 2 below). This means, in brief, that the intrinsic dynamics of the system are governed by the linear paradigm that small causes lead to small effects. Since linear equations can only lead to exponentially decaying (or growing) or (damped) periodically oscillating solutions, all irregular behaviour of the system has to be attributed to some random external input to the system. Now, chaos theory has taught us that random input is not the only possible source of irregularity in a system's output: nonlinear, chaotic systems can produce very irregular data with purely deterministic equations of motion in an autonomous way, i.e., without time dependent inputs. Of course, a system which has both, nonlinearity and random input, will most likely produce irregular data as well.
Although we have not yet introduced the tools we need to make quantitative statements, let us look at a few examples of real data sets. They represent very different problems of data analysis where one could profit from reading this book since a treatment with linear methods alone would be inappropriate.
Contents
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp v-x
-
- Chapter
- Export citation
Preface to the first edition
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp xi-xii
-
- Chapter
- Export citation
-
Summary
The paradigm of deterministic chaos has influenced thinking in many fields of science. As mathematical objects, chaotic systems show rich and surprising structures. Most appealing for researchers in the applied sciences is the fact that deterministic chaos provides a striking explanation for irregular behaviour and anomalies in systems which do not seem to be inherently stochastic.
The most direct link between chaos theory and the real world is the analysis of time series from real systems in terms of nonlinear dynamics. On the one hand, experimental technique and data analysis have seen such dramatic progress that, by now, most fundamental properties of nonlinear dynamical systems have been observed in the laboratory. On the other hand, great efforts are being made to exploit ideas from chaos theory in cases where the system is not necessarily deterministic but the data displays more structure than can be captured by traditional methods. Problems of this kind are typical in biology and physiology but also in geophysics, economics, and many other sciences.
In all these fields, even simple models, be they microscopic or phenomenological, can create extremely complicated dynamics. How can one verify that one's model is a good counterpart to the equally complicated signal that one receives from nature? Very often, good models are lacking and one has to study the system just from the observations made in a single time series, which is the case for most non-laboratory systems in particular. The theory of nonlinear dynamical systems provides new tools and quantities for the characterisation of irregular time series data.
5 - Instability: Lyapunov exponents
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 65-74
-
- Chapter
- Export citation
-
Summary
Sensitive dependence on initial conditions
The most striking feature of chaos is the unpredictability of the future despite a deterministic time evolution. This has already been made evident in Fig. 1.2: the average error made when forecasting the outcome of a future measurement increases very rapidly with time, and in this system predictability is almost lost after only 20 time steps. Nevertheless we claim that these experimental data are very well described as a low dimensional deterministic system. How can we explain this apparent contradiction?
Example 5.1 (Divergence of NMR laser trajectories). In Fig. 5.1 we show several segments of the NMR laser time series (the same data underlying Fig. 1.2; see Appendix B.2) which are initially very close. Over the course of time they separate and finally become uncorrelated. Thus it is impossible to predict the position of the trajectory more than, say, ten time steps ahead, knowing the position of another trajectory at this time which was very close initially. (This is very much in the spirit of the prediction scheme of Section 4.2.)
The above example illustrates that our every day experience, “similar causes have similar effects”, is invalid in chaotic systems except for short periods, and only a mathematically exact reproduction of some event would yield the same result due to determinism. Note that this has nothing to do with any unobserved influence on the system from outside (although in experimental data it is always present) and can be found in every mathematical model of a chaotic system.
This unpredictability is a consequence of the inherent instability of the solutions, reflected by what is called sensitive dependence on initial conditions.
I - Basic topics
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 1-2
-
- Chapter
- Export citation
14 - Coupling and synchronisation of nonlinear systems
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 292-303
-
- Chapter
- Export citation
-
Summary
The reason for the predominance of scalar observation lies partly in experimental limitations. Also the tradition of spectral time series analysis may have biased experimentalists to concentrate on analysing single measurement channels at a time. One example of a multivariate measurement is the vibrating string data [Tufillaro et al. (1995)] that we use in this book; see Appendix B.3. In this case, the observables represent variables of a physical model so perfectly that they can be used as state vectors without any complication. In distributed systems, however, the mutual relationship of different simultaneously recorded variables is much less clear. Examples of this type are manifold in physiology, economics or climatology, where multivariate time series occur very frequently. Such systems are generally quite complicated and a systematic investigation of the interrelation between the observables from a different than a time series point of view is difficult. The different aspects which we will discuss in this chapter are relevant in exactly such situations.
Measures for interdependence
As pointed out before, a first question in the analysis of simultaneously recorded observables is whether they are independent.
Example 14.1 (Surface wind velocities). Let our bivariate time series be a recording of the x-component and the y-component of the wind speed measured at some point on the earth's surface. In principle, these could represent two independent processes. Of course, a more reasonable hypothesis would be that the modulus of the wind speed and the angle of the velocity vector are the independent processes, and hence both x and y share some information of both.
B - Description of the experimental data sets
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 338-349
-
- Chapter
- Export citation
-
Summary
Throughout the text we have tried to illustrate all relevant issues by the help of experimental data sets, some of them appearing in several different contexts. In order to avoid repeats and to concentrate on the actual topic we did not describe the data and the systems they come from in any detail in the examples given in the text. This leeway we want to make up in this appendix, together with a list of all places where each set is referred to.
Lorenz-like chaos in an NH3 laser
This data set was created at the PTB Braunschweig in Germany in an experiment run by U. Hübner, N. B. Abraham, C. O. Weiss and collaborators (1993). Within the time series competition organised in 1992 by N. A. Gershenfeld and A. Weigend at the Santa Fe Institute it served as one of the sample series and is available on the SFI server by anonymous FTP to sfi.santafe.edu.
A paradigmatic mathematical model for low dimensional chaos is the Lorenz system, Lorenz (1969), describing the convective motion of a fluid heated from below in a Rayleigh–Benard cell. Haken (1975) showed that under certain conditions a laser can be described by exactly the same equations, only the variables and constants have different physical meaning. The experiment in Braunschweig was designed to fulfil the conditions of being describable by the Lorenz–Haken equations as closely as possible.
The time series is a record of the output power of the laser, consisting of 10 000 data items. Part of it is shown in Fig. B.1. Similarly to the Lorenz model, the system exhibits regular oscillations with slowly increasing amplitude.
9 - Advanced embedding methods
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 143-173
-
- Chapter
- Export citation
-
Summary
The reconstruction of a vector space which is equivalent to the original state space of a system from a scalar time series is the basis of almost all of the methods in this book. Obviously, such a reconstruction is required for all methods exploiting dynamical (such as determinism) or metric (such as dimensions) state space properties of the data. In the first part of the book we introduced the time delay embedding as the way to find such a space. Because of the outstanding importance of the state space reconstruction we want to devote the first section of this chapter to a deeper mathematical understanding of this aspect. In the following sections we want to discuss modifications known as filtered embeddings, the problem of unevenly sampled data, and the possibility of reconstructing state space equivalents from multichannel data.
Embedding theorems
A scalar measurement is a projection of the unobserved internal variables of a system onto an interval on the real axis. Apart from this reduction in dimensionality the projection process may be nonlinear and may mix different internal variables, giving rise to additional distortion of the output. It is obvious that even with a precise knowledge of the measurement process it may be impossible to reconstruct the state space of the original system from the data. Fortunately, a reconstruction of the original phase space is not really necessary for data analysis and sometimes not even desirable, namely, when the attractor dimension is much smaller than the dimension of this space.
3 - Phase space methods
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 30-47
-
- Chapter
- Export citation
-
Summary
Determinism: uniqueness in phase space
The nonlinear time series methods discussed in this book are motivated and based on the theory of dynamical systems; that is, the time evolution is defined in some phase space. Since such nonlinear systems can exhibit deterministic chaos, this is a natural starting point when irregularity is present in a signal. Eventually, one might think of incorporating a stochastic component into the description as well. So far, however, we have to assume that this stochastic component is small and essentially does not change the nonlinear properties. Thus all the successful approaches we are aware of either assume the nonlinearity to be a small perturbation of an essentially linear stochastic process, or they regard the stochastic element as a small contamination of an essentially deterministic, nonlinear process. If a given data set is supposed to stem from a genuinely non-linear stochastic processes, time series analysis tools are still very limited and their discussion will be postponed to Section 12.1.
Consider for a moment a purely deterministic system. Once its present state is fixed, the states at all future times are determined as well. Thus it will be important to establish a vector space (called a state space or phase space) for the system such that specifying a point in this space specifies the state of the system, and vice versa. Then we can study the dynamics of the system by studying the dynamics of the corresponding phase space points. In theory, dynamical systems are usually defined by a set of first-order ordinary differential equations (see below) acting on a phase space.
A - Using the TISEAN programs
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 321-337
-
- Chapter
- Export citation
-
Summary
In this chapter we will discuss programs in the TISEAN software package that correspond to algorithms described in various sections throughout this book. We will skip over more standard and utility routines which you will find documented in most software packages for statistics and data analysis. Rather, we will give some background information on essential nonlinear methods which can rarely be found otherwise, and we will give hints to the usage of the TISEAN programs and to certain choices of parameters.
The TISEAN package has grown out of our efforts to publicise the use of nonlinear time series methods. Some of the first available programs were based on the code that was printed in the first edition of this book. Many more have been added and some of the initial ones have been superseded by superior implementations. The TISEAN package has been written by Rainer Hegger and the authors of this book and is publicly available via the Internet from http://www.mpipks-dresden. mpg.de/∼tisean
Our common aim was to spare the user the effort of coding sophisticated numerical software in order to just try and analyse their data. There is no way, however, to spare you the effort of understanding the methods. Therefore, still none of the programs can be used as a black box routine. Programs that would implement all necessary precautions (such as tests for stationarity, estimates of the minimal required number of points, etc.) would in many cases refuse to attempt, say, a dimension estimate. But, even then, we suspect that such a library of nonlinear time series analysis tools would rather promote the misuse of nonlinear concepts than provide a deeper understanding of complex signals.
Frontmatter
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp i-iv
-
- Chapter
- Export citation
10 - Chaotic data and noise
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 174-196
-
- Chapter
- Export citation
-
Summary
All experimental data are to some extent contaminated by noise. That this is an undesirable feature is commonplace. (By definition, noise is the unwanted part of the data.) But how bad is noise really? The answer is as usual: it depends. The nature of the system emitting the signal and the nature of the noise determine whether the noise can be separated from the clean signal, at least to some extent. This done, the amount of noise introduces limits on how well a given analysing task (prediction, etc.) can be carried out.
In order to focus the discussion in this chapter on the influence of the noise, we will assume throughout that the data are otherwise considerably well behaved. By this we mean that the signal would be predictable to some extent by exploiting an underlying deterministic rule – were it not for the noise. This is the case for data sets which can be embedded in a low dimensional phase space, which are stationary and which are not too short. Violation of any one of these requirements leads to further complications which will not be addressed in this chapter.
Measurement noise and dynamical noise
When talking about noise in a data set we have to make an important distinction between terms. Measurement noise refers to the corruption of observations by errors which are independent of the dynamics. The dynamics satisfy xn+1 = F(xn), but we measure scalars sn = s(xn) + ηn, where s(x) is a smooth function that maps points on the attractor to real numbers, and the ηn are random numbers.
4 - Determinism and predictability
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 48-64
-
- Chapter
- Export citation
-
Summary
In this chapter we will discuss the notion of the predictability of a system evolving over time or, strictly speaking, of a signal emitted by such a system. Forecasting future values of some quantity is a classical problem in time series analysis but the conceptual importance of the prediction problem is not limited to those who want to get rich by knowing tomorrow's exchange rates. Even if, instead, you are interested in describing, understanding or classifying signals, stay with us for a few pages.
In this book we are concerned with the detection and quantification of possibly complicated structures in a signal. We want to be able to convince others that the structures we find are real and not just fluctuations. The most convincing argument for the presence of some pattern is if it can be used to give an improved prediction. It is a necessary condition for a theory to be compatible with the known data but it is not sufficient. In order to become accepted, a theory must successfully predict something which can be verified subsequently. In time series analysis, we can take this requirement of predictive quality quite literally.
Most concepts, which we will introduce later in order to describe time series data, can be interpreted to some extent as indirect measures of predictability. Due to their indirect nature, some conclusions will remain controversial, especially if the structures are rather faint. The statistically significant ability to predict the signal better than other techniques do will then be a more convincing affirmation of nonlinear and deterministic structure than several dubious digits of the fractal dimension.
Index
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 365-369
-
- Chapter
- Export citation
7 - Using nonlinear methods when determinism is weak
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 105-125
-
- Chapter
- Export citation
-
Summary
In the preceding two chapters we established algorithms to estimate the Lyapunov exponent and the correlation dimension from a time series. We tried to be very strict about the conditions which must be met in order to justify such estimates. The data quality and quantity had to be sufficient to observe clear scaling regions. The implied requirement that the data must be deterministic to a good approximation is also valid for successful nonlinear predictions (Chapter 4). If this were the whole story, the scope of these methods would be quite limited. In the main, well-controlled laboratory data from experiments which have been designed to show deterministic chaos would qualify. Although these include some very interesting signals, many other data sets for which classical, linear time series methods seem inappropriate do not fall into this class.
Indeed, there is a continuous stream of publications reporting more or less successful attempts to apply nonlinear algorithms, in particular the correlation dimension, to field data. Examples range from population dynamics in biology, stock exchange rates in economy, and time dependent hormone secretion or ECG and EEG signals in medicine to geophysical records of the earth's magnetic field or the variable luminosity of astronomical objects. In particular the interpretation of the results as measures of the “complexity” of the underlying systems has met with increasing criticism. It is now quite generally agreed that, in the absence of clear scaling behaviour, quantities derived from dimension or Lyapunov estimators can be at most relative measures of system properties. But even then it is not clear which properties are really measured.
11 - More about invariant quantities
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 197-233
-
- Chapter
- Export citation
-
Summary
In the first part of this book we introduced two important concepts to characterise deterministic chaos: the maximal Lyapunov exponent and the correlation dimension. We stressed that one of the main reasons for their relevance is the invariance under smooth transformations of the state space. Irrespective of the details of the measurement process and of the reconstruction of the state space, they will always assume the same values. Of course, this is strictly true only for ideal, noise-free and infinitely long time series, but a good algorithm applied to an approximately noise-free and sufficiently long data set should yield results which are robust against small changes in the parameters of the algorithm.
The maximal Lyapunov exponent and the correlation dimension are only two members of a large family of invariants, singled out mainly because they are the two quantities which can best be computed from experimental data. In this chapter we want to introduce a more complete set of invariants which characterises the stability of trajectories and the geometrical and information theoretical properties of the invariant measure on an attractor. These are the spectrum of Lyapunov exponents and the generalised dimensions and entropies. These quantities possess interesting interrelations, the Kaplan–Yorke formula and Pesin's identity. Since these relations provide cross-checks of the numerical estimates, they are of considerable importance for a consistent time series analysis in terms of nonlinear statistics.
Preface to the second edition
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp xiii-xiv
-
- Chapter
- Export citation
-
Summary
In a field as dynamic as nonlinear science, new ideas, methods and experiments emerge constantly and the focus of interest shifts accordingly. There is a continuous stream of new results, and existing knowledge is seen from a different angle after very few years. Five years after the first edition of “Nonlinear Time Series Analysis” we feel that the field has matured in a way that deserves being reflected in a second edition.
The modification that is most immediately visible is that the program listings have been be replaced by a thorough discussion of the publicly available software TISEAN. Already a few months after the first edition appeared, it became clear that most users would need something more convenient to use than the bare library routines printed in the book. Thus, together with Rainer Hegger we prepared stand-alone routines based on the book but with input/output functionality and advanced features. The first public release was made available in 1998 and subsequent releases are in widespread use now. Today, TISEAN is a mature piece of software that covers much more than the programs we gave in the first edition. Now, readers can immediately apply most methods studied in the book on their own data using TISEAN programs. By replacing the somewhat terse program listings by minute instructions of the proper use of the TISEAN routines, the link between book and software is strengthened, supposedly to the benefit of the readers and users. Hence we recommend a download and installation of the package, such that the exercises can be readily done by help of these ready-to-use routines.
Nonlinear Time Series Analysis
- 2nd edition
- Holger Kantz, Thomas Schreiber
-
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003
-
The paradigm of deterministic chaos has influenced thinking in many fields of science. Chaotic systems show rich and surprising mathematical structures. In the applied sciences, deterministic chaos provides a striking explanation for irregular behaviour and anomalies in systems which do not seem to be inherently stochastic. The most direct link between chaos theory and the real world is the analysis of time series from real systems in terms of nonlinear dynamics. Experimental technique and data analysis have seen such dramatic progress that, by now, most fundamental properties of nonlinear dynamical systems have been observed in the laboratory. Great efforts are being made to exploit ideas from chaos theory wherever the data displays more structure than can be captured by traditional methods. Problems of this kind are typical in biology and physiology but also in geophysics, economics, and many other sciences.
II - Advanced topics
- Holger Kantz, Max-Planck-Institut für Physik komplexer Systeme, Dresden, Thomas Schreiber, Max-Planck-Institut für Physik komplexer Systeme, Dresden
-
- Book:
- Nonlinear Time Series Analysis
- Published online:
- 06 July 2010
- Print publication:
- 27 November 2003, pp 141-142
-
- Chapter
- Export citation